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ABSTRACT 

Institutional research and the role of Information 
systems are examined and three types of information provided by 
Institutional research offices are discussed: institutional studies, 
management analyses, and periodic management reports. The varying 
characteristics of research offices are described and the role of the 
institutional research office In the decision -making process ir. 
examined. It Is noted that increasingly the large, cocputerlzed 
Information systems are the principal source for Information and 
dominate administrative data processing. Specific functions of these 
systems are discussed and it is emphasized that Integrated 
information systems may be the most useful for higher education 
institutions. Various Impacts of information systems on institutional 
research are also reviewed, including Issues cf data guallty, control 
of the system, and the degree of system Integration. It is suggested 
that effective utilization of infcrmatlon systems centers on 
communication between processors and analysts. The need for 
institutional Information systems to adapt to the rapidly changing 
climate of academe is emphasir-ed. The role of on-line data base 
systems is discussed and the impact of state and national information 
systems is examined. It is concluded that for colleges and 
universities the institutional research office should become the 
planning system by aaintaining an accurate, accessible, acd 
integrated data base. (SF) 
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Rapid developments of increasingly complex information 
systems are having considerable effect on institutional research. 
Institutional research offices are sophisticated users of computer 
systems as well as gatherers of data directly through question- 
naires, surveys, and non-mechanized records. What is the im- 
pact of the increased use of computer systems on the practice of 
institutional research? What does the analyst need to look for in 
using information systems? What peculiar problems are pre- 
sented in using someone else*s data? What are the prospects for 
the future, given rapid cechnological change? Discussion of 
these and related questions requires a brief overview of the 
function of institutional research offices, the context in which 
they operate, and the data they u::;e. 

Functions of Institutional Research 

Institutional research offices provide three kinds of infor- 
mation: institutional studies, management analyses, and 
periodic management reports. Institutional studies are detailed 
reviews of particular a.spects of institutional activity and are 
often designed to answer questions raised by outside agencies 
such as accrediting bodies. Manaficnient analyses can be insti- 
gated either by the analyst or the institutional executive but are 
designed to answer questions of immediate importance to the 
institution and its managers. (How many faculty will retire in 
1980? What is the impact of raising retirement age of faculty? If 
admissions requirements are raised 10% for freshmen, what will 
be the effect on the number of students admitted? How do faculty 
salaries compare with those of other institutions?) Periodic 
manaf^emvnt reports arc ongoing, regularly recurring reports 
such as those on faculty load, student crcdii hours taught, the 
cost of student credit hours produced, decrees grunted, and 
students enrolled. 

Constraints Surrounding the Institutional Research Office 

The institutional research office does not operate in a 
vacuum, and the reporting arrangements, lines of communica- 
lion, and information flow vary substantially from institution to* 
institution. In some institutions, institutional research offices are 
research centers located outside the ad linistrative chain of 
deans, vice presidents, and president. These offices tend to 
concentrate on long-term studies and regular reporting. Their 
techniques and methodologies arc derived from psychology and 
other social sciences, and their results are published often in 
various journals. 

The institutional research office may also be located very 
near the president or provost and may be the central staff office 
for answering questions about the functioning of the institution . 
Proximity to top management insures a continuous flow of 
requests requiring rapid response. 

In addition to the organizational location, it is important to 
know the particular climate surrounding the institutional re- 
search office. A characteristic of th^: institutional research unit 
located near the institutional executive is that its products arc 
always needed yesterday. Institutional gnvemanco is sometimes 
characteri;:ed as organized anarchy, with executives moving 



from one crisis to another, with little or no time for reflection, 
planning, or studied decision making. Failure of the institutional 
research analyst to respond quickly in such a setting usually 
results in the executive turning to someone else for answers. 
Another constraint 011 the institutional research function is 
! there is always a variety of problems and issues under 
consideration at any one time. An institutional research office in 
a lai^ge university may t)e considering tuition increases and 
faculty salaries, planning for enrollment downturns, reviewing 
new programs in the health sciences, answering legislative 
inquiries about what faculty do with their time, and doing all 
within a short period of time. The scope of questions requires the 
analyst to have access to a wide variety of information covering 
all aspects of the institution. 

Decision-Making Style 

Much of American higher education is characterized by a 
participative style of management in which governance is shared 
by faculty committees, faculty-administration task forces, fac- 
ulty senates, governing boards, committees of deans, affiliated 
deparment chairmen, and advisory groups. Decision making 
tends to involve a number of groups. Some of these groups have* 
formally designated powers and responsibilities so that particu- 
lar proposals must be approved by them; faculty oversight of 
course and graduation requirements is an example . Other groups 
hdvc fev/er formal responsibilities or may be formed only for a 
special study, such as an ad hoc task force on campus par';ing 
problems. Some groups have a stable membership but a chang- 
ing agenda, for instance an executive committee of faculty who 
advise the president on academic policy or budgetary matters. 

The existence of a number of groups within an institution 
has several implications for the style of decision making and, 
htnce, the style of analysis that is most viable. Many of the 
groups that are formally or informally consulted on a particular 
issue have little in the way of background information but 
usually have strong opinions about what the relevant variables or 
method of analysiv. may be. 

Faculty members are automatically experts on the adminis- 
tration of an institution and may have particular "pet** ap- 
proaches that they want incorporated in the analysis The analyst 
must be prepared, then, to meet the demands of different 
audiences, to Mructure and restructure the analysis to take 
account of d'* .ent poims of view or to include new factors in 
the analysis l acuity salaries, for instance, are of compelling 
concern to faculty groups. Faculty will usually have notions 
about an appropriate comparison group as well as appropriate 
methods of analysis. The analyst must be prepared to change 
comparison data to he sure that all appropriate institutions are 
included or to accommodate suggestions made by different 
faculty representatives. The kmd of comparison— whether 
rank -by-rank averages, for cimtinuing faculty, or new and con- 
tinuing f aculty, or aggregated averages, whether weighted rank 
hy mix or not— w'll be of concern also. While it is helpful, 
initially, to spend time with all the advisory groups so that the 
preliminary analysis can be as complete as possible, it is impos- 
sible in most cases to anticipate every question in advance or 
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include every permutation of the data. Operating in an environ- 
ment in which the audience is changing its formulation of the 
^nuestion requires the analyst to have the ability to analyze and 
i%.v. iaiyze the data in a variety of ways, often with little advance 
warning. 

The institutional research analyst, as a result of the special 
work environment of institutional research, must work under 
great time pressure, must contend with a large volume of 
pressing concerns, each competing for attention, and must have 
access to and be knowledgeable about sources of information 
within and without the institution. The analyst must do all this 
rapidly, accurately, and with good humor 

'Kchniques of Inslilulional Research 

The techniques and resources that the institutional re- 
searcher uses include statistical methods, information from 
institutional information systems ^nd from other institutions, 
assembly of historical data, analytical methods derived from 
operations research and economic theory, and data gathered 
through questionnaires, surveys, and other direct methods. 

Statistical techniques allow the analyst to analyze informa- 
tion and to determine its limits as well as its reliability. By 
drawing on small sample theory, regression analysis, and other 
tests, the analyst can answer the question How good are these 
data? or How can they be extended? This is vital for policy 
analysis purposes. A president, for example, needs to know how 
much to trust an estimate that enrollment will be 69c greater next 
year because institutional decisions of some magnitude hinge on 
the answer to that question. 

Comparative information gathered directly from other in- 
stitutions, or from nationally available data banks supported by 
large information systems, is an increasingly important tool for 
the institutional research analyst. Institutions do not exist in a 
vacuum, and comparisons of salaries, tuition and fees, enroll- 
ment trends, financial standings, and the quality of graduate 
programs are being made formally and intbrmally Knowledge 
of and access to national data banks about higher education is 
important in order to supply answers to questions about what 
other in.stitutions are doing. Contacts with analysts at similar 
institutions, either formally oi informally, permit the develop- 
ment of shared information not available nationally 

Historical data bases for the analyst's institution, as well as 
for other comparative institutions, are another resource in the 
analyst 's repertoire. Regular management reports are often a part 
of the data bases of the institutional research ofhce. These can be 
compiled and naintained so that questions about trends can be 
answered. Projection is difficult without adequate historical data 
bases, and routine review of trends is an essential part of 
management reporting. 

Analytical and modeling methods ate used to examine 
*iypothetical questions: What if enrollment stays the same next 
year? What if a new progri^m is added to the health profession 
area? What if mos; faculty choose to take early retirement? What 
if the trend towards vocational subjects continues? Institutional 
research offices have models of student demand for courses and 
translate these into requirements for faculty. Models of faculty 
entry and exit from the work force and student How models are 
utilized also. .Selection of the appropriate model and the choice 
of data to estimate the parameters of the model are steps in 
answering the "what if " questions. 

Institutional research analysts, in both their research and 
policy analysis roles, have often gathered information directly 
from surveys, questionnaires, interviews, or Delphi instru- 
ments. Standard instruments have been developed for many 
topics, and survey researcli itself is now a specialized siihficld 
within sociology. Techniques for developing and administering 
questionnaires to insure reliable results arc highly developed, 
and can be obtained often from specialized liniiN or groups that 
exist to conduct survey research 



All of these techniques are dependent on information, and 
for the institutional research analyst, the principal source for 
information about the institution is, increasingly, the large, 
computerized information systems that dominate administrative 
data processing units at most campuses. 

Characteristics of Computerized Information Systems 

Given that much of the data used in institutional research 
analysis is processed by these lscf;c systems, what are the 
characteristics of computerized systems, and what is their im- 
pact on the kind of analysis that can be done, its accuracy, 
validity, and acceptability? 

Most information systems used in our institutions are 
operational in nature, that is, they exist to serve specific day-to* 
day purposes. Payroll and personnel information systems exist 
primarily to process payroll checks and keep track of who is 
getting paid how much. Accounting systems have as a primary 
function keeping track of expenditures by budget category tnd 
the preparation of fiscal reports and annual statements. Budget 
systems exist to report tk status of budgets, and equipment 
inventory .systems record the location, value, and disposition of 
equipment. There are also a few systems which have as their 
primary purpose supplying management or planning informa- 
tion. (These will be discussed toward the end of the paper.) The 
bulk of the information used by the institutional research analyst 
comes from .systems that were not designed with analytical or 
management problems in mind. 

Institutional information systems are also complex. In their 
complexity they aa* rigid and formalized, with a set of proce- 
dures for entering data, a processing schedule that satisfies the 
principal operational user, and a set of standards for the opera* 
tion of the system. Operational systehis are usually well 
documented simply because they are operational, but their 
complexity may make them difficult to understand or manipu- 
late. 

Rigidity is associated both with the complexity and opera* 
tional nature of most information systems. Because p.iyrtill 
checks need to be produced ev;;ry month, or accounts balanced 
at the end of the year, or purchases paid for, the systems n^ ist be 
predictable and stable. The operational demands of the system 
do not permit frequent changes, and, once established, modifica* 
tions can only be made in a systematic fashion so that the 
underlying work uf the system is not compromised. 

\ rather unfortunate aspect of most institutional informa- 
tion systems is that they are unintegrated. Bach system develop- 
ment project is a separate activity and, as new systems are added 
or old ones modified, the latest technological and processing 
innovations are introduced. Often, however, limits on the re- 
sources available do not permit systems designers to go back and 
update all the existing systems to reflect the latest technology and 
design innovations. Because operational systems are developed 
at different points in time, the data definitions carried on the 
various systems may vary substantially There tends to '» ; some 
ovei lap in thf* information that is recorded on systems, often with 
slight variations in definitions. Faculty ap^Kiintments recorded 
on an academic personnel system may include persons who arc 
appointed to faculty rank but vvfio are not getting paid from the 
institutions, such as doctors who dc.natc thi ir time to medical 
schools or businessmen who yive occasional lectures. Payroll 
counts do not include these "free" faculiy Thus, a count of 
faculty based on the academic personnel system will overlap but 
not agree with the count of faculty based on those individuals 
who are receiving paychecks from the institution. 

Because systems are unintegrated, the analyst who aggre- 
gates data from several different systems faces trouble. A 
ilvinmmvnt. as defined on the budget system, is a collection of 
budget or account codes, while a department in the student 
registration s\stcm may be a collection of student major or 
course identifiers. A department for the physical space inventory 
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system is a collection of rooms, or even a building or two, Which 
department is thr analyst to use, and how are the varying 
definitions to be reconciled? 

An additional characteristic of institutional information 
systems that may cause trouble for the analyst derives from their 
operational nature — someone other than the analyst user con- 
trols the system. Not only does this mean that the system is 
seldom designed with the analyst *s needs in mind, but access to 
the system is limited and the right to initiate changes is almost 
nonexistent. The operational aspect of the system must be 
maintained at all costs, and changes can be made only when they 
do not interfere with the operation. Also, the principal opera- 
tional user must be convinced that the changes or ^idditions 
sought by the analyst are important and necessary. 

fmpact on Institutional Research 

How do these characteristics affect institutional research? 
The operational nature of the information system has, perhaps, 
the most direct impact on the potential uses of the data by the 
analyst. Data quality is a concern of both the analyst and the 
operational u.ser. However, the concern with dati quality is 
related directly to ihe operational purpose of the system. The 
payroll manager wants to be sure that checks in correct amounts 
are issued to appropriate p rsons and that the correct deductions 
are taken from gross pay ami recorded in a systematic way. The 
payroll managei, though, may be less concerned about whether 
the address recorded on the payroll record is correct if the 
address is not used to distribute paychecks. Attempts by the 
analyst to use this address to study the transportation needs of 
institutional employees may be less than satisfactory. Or, the 
payroll record foi an individual may have the person*s highest 
degree earned entered when the individual is hired. An analyst 
who depends on that record to analyze salary by degree held may 
find that all degrees earned subsequent to employment have not 
been entered. The payroll manager is not concerned with correct- 
ing that information since it dues not affect payrolls. Data quality 
is uneven in porational systems since there is a tendency for the 
operational manager to audit and review data only in those areas 
which affect the principal function of the system. Higher quality 
data exist when there is a built-in audit check. The payroll 
system, for instance, has such a check when employees receive 
their paychecks and see that the dollar amount is correct. The 
more peripheral data are to the central purpose of the system, the 
lower their quality tends to be. 

Another aspect of an operational system which causes 
concern for the analyst has been alluded to in the discussion of 
the control of the system. Ciianges to operational .systems are 
slow, time consuming, and difficult to make. The analyst often 
must face the question of whether it is worth the effort to get a 
sy.stem changed so that data quality can be improved. For most 
purposes, the data as reported on the operational systems are 
what the analyst has to work with and, given demands for rapid 
response, the analyst has no choice but to try somehow to adapt 
the exi.sting data to fit the problem being examined. 

The unintegrated nature of information systems means that 
considerable analytical effort is spent in trying to determine what 
IS the most correct definition to use and which data system i.s the 
best source for the desired information. Correct really has two 
senses that arc relevant for the analyst: in the most elementary 
case, the analyst needs data that have been accurately entered, 
updated, and recorded and which are processed correctly by 
computer programs. Incorrect data, in this sense, exist in every 
system, even when the analyst collects the basic data. A gixjd 
analytical sense and a feel tor the underlying realities being 
measured, a. well as .some experience with the results of similar 
kinds of an;ilys.^s, are among the techniques the analyst must use 
to work with this problem. 

A more serious kind of problem affects the usiir of uninte- 
grated information .systems. The institutional researcher is often 



one of the few persons in the institution who ass'^mbles data (torn 
a variety of information systems, Where systems are uninte- 
grated, definitions often do not agree, and the analyst must 
decide how to fit the competing definitions together. An analyst, 
"for example, may^needtoit^sembte an inventory of the physical" 
space a department occupies to determine if there is adequate 
space for all faculty paid by the department. The physical 
in veiitory file may record the location of every class taught by the 
department, and the payroll mpy record all the faculty paid by the 
department budget; yet-many institutions assign courses to the 
department offering the course, rather than to the department 
paying the faculty. If significant numbers of non-drpartment 
facuky are teaching courses, the physical inventory may give a 
misleading picture of the square feet occupied by the faculty paid 
by the department. The potential link between the two 
systems — the definition of a department — is missing in both 
systems, and the analyst must supply it. 

Since the systems were not constructed with potential 
analytical needs in mind, consistent definitions often canr.ot be 
used, and the analysiscan only be approximate. Etefinitions used 
in an operational system may be appropriate to the needs of that 
system, and the operational user may be reiuctant to change to 
meet the needs of the analyst. The fact that the analyst is, at best, 
a secondary user means that the likelihood of change is not high 
unless the operational user can be convinced of the pressing need 
for consistency. 

The question for the analyst, then, is how to manage a 
situation in which most of the information is supplied by a 
computerized system that is not susceptible to change, where the 
demand is for immediate response, and where the analyst has 
little or no control over the infomation system which contains 
data of uneven quality. 

Rffective Institutional I ^search Operation 

A key to effective institutional research analysis in this 
environment becomes the lines of communication that the 
analyst maintains, both to the data processing group and the 
operational system managers. To the extent that the analyst can 
make wi.shes known when .system changes are contemplated, or 
to the extent that proposed changes the system are reviewed by 
th^ analyst, the operational systei.« can be modified so that the 
analyst can serve llie executive more effectively. There is, 
however, a limited ability to change systems, in that the prob- 
lems facing the executive change faster than systems can be 
changed. No matter how good the crystal ball of the analyst and 
executive the next iteration of a particular problem always seems 
to be slightly different from the last one Thus, what was created 
to provide the appropriate information the recent par.t is not 
quite satisfactory today. 

Organizationally, the conflict between the needs of stable 
operational systems and the needs of the analyst has led to the 
almost mandatory "in-house" data processing capability in 
institutional re.serach offices. Analyst-programmers, who are 
familiar with all of the operational systems, their data dehni- 
tions. and the techniques for extracting and cross-matching data 
from several systems, often make the difference in answering 
executive questions. These analyst-programmers may have rela- 
tively little time for careful documentation, as desirable as that 
is. but they are expert at what have come to be known as ''quick 
and dirty" programs. These are programs that quickly teil the 
analyst, for example, how many faculty would be affected by a 
proposed pay limitation from the National Science I oundation 
or how many entering freshmen came from high .schcK)ls within 
commuting range of the institution. 

Institutional researchers have to know how to work with 
imperfect data and to cross-check anu reconcile data from 
competing sources. Differences between systems must be ac- 
counted for, and the analyst must be prepared to defend the 
particular data source chosen to answer the question. Dcfini- 
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tional di^eren^es between systems can be particularly important 
when the issue is one that will concern a variety of consti(M'*n* 
cies, since altering the definition may meet the needs of one 
group more than another 

Prospects for Change 

Are there potential changes to information systems that are 
going to make the analyst s task easier? Information systems has 
been an area where the pace of technical change has been very 
rapid, and new developments have revolutionized processing; 
and design requirements. Software and hardware changes have 
expanded the capability of information systems both in terms of 
size and speed . Perhaps the development of most potential use to 
the analyst will be the widespread adoption of easily used, 
on-line data ba«e systems, lb the extent that the institutional 
researcher can interact directly with the data, the problems of 
definition, timeliness, and lack of integration can be minimized. 
However, it would be easy, and probably misleading, to think 
that all ptublems arc going to be solved by technological 
changes. Theie are several factors that suggest that data prob- 
lems will be with us for some years to come: (I) there will be 
limited funds available for extensive adoptions of new systems 
or .lew hardware, given declining enrollments and relatively 
weakeningstateand federal support for higher education; institu- 
tions will have to maintain their existing systems for some time; 
(2) technological -^^ange has been oversold in the past as the 
answer to managemt.it problems. Higher education institutions 
are inherently complex organizations with uncertain goals and 
directions, operating in a relatively rapidly changing political 
and social climate. Any technological improvement, or new 
system or equipment fei»*ure, may answer management ques- 
tions at the time it is installed. Yet. the institution will not remain 
static, and new needs will emerge. 

Impact of State and National S> stems 

The discussion has focused on institutional information 
systems and the analyst as a user of ir stitutionally generated 
information A further important con' Ideration is (hat of the 
effect of state and national information systems and their data 
bases on institutiona! research. 

State and national systems present many of the same 
problems for institutional reserach offices as the institutional 
operational systems. The definitions required for the national 
reports may not be the same as those used internally, and 
reconciliation probiems arise. Control of the larger system does 
not rest with the analyst, who can only be a user, adapting (he 
data to his or her own needs. State and national data systems 
obviously broiden the opportunities for analysis, and make 
possible a mu jh wider variety of comparative .studies. Cro.ss- 
institutional comparisons are of widespread interest but have 
many difficulties. Some nationally developed procedures, such 
as the NCHFMS Information Exchange Procedures, have at- 
tcr ipted to define carefully a set of definitions and prcKcdures 
\» ithin which uniform data can be exchanged. Since institutions 
/ary widely in their organization and information systems, these 
procedures require careful, detailed study to be implemented. 
Eiven then, data comparisons may be ditiuult. Some of the more 
promising work in interin.stitutional comparative data has been 
done by trying tn understand the detailed structure ot other 
institutions and then casting (hat information in lemis that are 
relevant to the analysts institution. 

Plannin)4 and Manaf^ement Systems 

The special rigor of institutional research suggest (hat 
analysis and system designers should develop special pur^ nse 
planning: or executive data systems, designed to meet the needs 
o\ the analyst and institutional executive. These systems would 
have data consis(enll> defined, integrated, ami at a level ol 



aggregation appropriate to the manager or analyst. They would 
draw from existing operational systems, but that dau would be 
designed to answer the kind * of questions most oftrn posed by 
the manager. 

SuetHevelopments might well resolve a nunber .'ques- 
tions raised by the existing interface between opeivitional sys- 
tems and management needs, but experience suggests some 
caution. There are several reasons for this: Any planning or 
executive system that maintains records separate from an opera- 
tional system will have reconciliation problems. The data items 
included will represent the analyst *s and executive's view of 
what are important and relevant data at a given point in time . In a 
rapidly changing environment, it may not be possible to specify 
a set of categories general enough to encompass all of the 
problems that the executive or analyst wiU need to address in the 
coming years. Finally, the level of aggregation is very difficult to 
specify in designing such a system. In many cases, aggregated 
information is all that is needed, yet there are occasional cases 
where detailed information is required. Designing any one 
system to include bcth those levels may be duplicative ano 
cumbersome. 

The Role of the Institutional Research Office 

Rather than designing ehe peifect planning information 
system, a more sensible alternative would be for institutional 
research offices to become the planning system. V/ith a knowl- 
edgeable group of analysts and analyst-programmers and good 
access to institutional information systems, most one-time re- 
ports can be produced relatively rapidly. Then, after manage- 
ment review and consideration, additions can be designed. After 
several years of requests for comparative salary reports, for 
instance, a regular procedure may be instituted and reports 
produced at .scheduled intervals. Frequent interaction with 
executives, as they define their own data needs, will insure that 
their needs are heard; changes can be made as circumstances and 
problems change. 

What conclusions, then, can be drawn about the relation- 
ship beiween the institutional research function and information 
systems? Information systems, as they exist now withih institu- 
tions and in .state and national settings present significant prob- 
lems to the analyst who must cope with less than adequate data. 
However, the systems have become .so endemic that it is usually 
easier and cheaper to use existing sy.stems and modify exi.sting 
data than to begin anew. By knowing wnat is collected , including 
the detailed definitions and the methods by which the data are 
updated and checked, the analyst can decide how they should be 
adapted and modified to an.swerthe policy questions. There is no 
substitute for knowledge of these systems in trying to provide 
analysis and data to institutional executives. 

The institutional research office must also be involved in 
system review and development to reduce the problems involved 
in using the data in the future and to insure that management s 
needs for information can be satisfied. 
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